Image processing device, image generation device, image processing method, and image generation method

ABSTRACT

An embodiment of this invention is an image processing device, including: a sectioning unit for sectioning an image into a plurality of sections to be processed; and a classification unit for classifying the plurality of sections to be processed into at least a clarification-processing-adapted area for embedding a latent image into the image, and an obfuscation-processing-adapted area for embedding a digital watermark into the image based on a characteristic of the image.

CLAIM OF PRIORITY

The present application claims priority from Japanese patent applicationJP 2011-116695 filed on May 25, 2011, the content of which is herebyincorporated by reference into this application.

BACKGROUND OF THE INVENTION

This invention relates to a technology for classifying an image into aplurality of areas based on characteristics of the image.

There has been a problem of improper leakage of moving images to theoutside of an organization. Particularly, improper leakage of movingimages via video-sharing websites on the Internet has broad influence,and thus tends to be a problem. Improper video posting to video-sharingwebsites includes an intentional posting by a convinced criminal as wellas a careless posting by a general person, both of which have beenproblems.

There has conventionally been known a method involving embedding an IDand the like identifying a distribution destination in a moving image bymeans of the digital watermark, and displaying this fact whendistributing the moving image, thereby restraining the posting itself.US 2001/0012019 discloses a method involving embedding a visible digitalwatermark such as logo and an invisible digital watermark into an image.

SUMMARY OF THE INVENTION

However, in the method described in US 2001/0012019, an area is simplydivided independently of a characteristic of the image to be subjectedto embedding, and there is thus a problem that an unauthorized user canremove the invisible digital watermark.

This invention has been created in view of the above problem and anobject of this invention is to provide a technique to classify sectionsof an image into a clarification-processing-adapted area for embedding alatent image into the image, and an obfuscation-processing-adapted areafor embedding a digital watermark into the image based on acharacteristic of the image.

An aspect of this invention is an image processing device, including: asectioning unit for sectioning an image into a plurality of sections tobe processed; and a classification unit for classifying the plurality ofsections to be processed into at least aclarification-processing-adapted area for embedding a latent image intothe image, and an obfuscation-processing-adapted area for embedding adigital watermark into the image based on a characteristic of the image.

An aspect of this invention allows classifying sections of an image intoan area appropriate for clarification-processing and an area appropriatefor obfuscation-processing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a system configuration of an imagegeneration device according to an embodiment of this invention.

FIG. 2 is a diagram illustrating an example of an internal configurationof a general PC.

FIG. 3 is a flowchart illustrating a process flow carried out in animage dividing device.

FIG. 4 is a flowchart illustrating an overview of sectioning processingin space direction, which is an example of image sectioning processingcarried out by the image dividing device.

FIG. 5 is a flowchart illustrating an overview of sectioning processingin time direction, which is an example of the image sectioningprocessing carried out by the image dividing device.

FIG. 6 is a flowchart illustrating an overview of sectioning processingin time/space direction, which is an example of the image sectioningprocessing carried out by the image dividing device.

FIG. 7 is a flowchart illustrating an overview of frequency analysisprocessing as image partial characteristic analysis processing carriedout in an image partial characteristic analysis device.

FIG. 8 is a flowchart illustrating an overview of template analysisprocessing as the image partial characteristic analysis processingcarried out in the image partial characteristic analysis device.

FIG. 9 is a flowchart illustrating a process flow carried out in theimage generation device.

FIG. 10 is a table showing various examples of information forclarification processing according to the embodiment of this invention.

FIG. 11 is a table showing various examples of information forobfuscation processing according to the embodiment of this invention.

FIG. 12 is a table showing an example of section information in a caseof sectioning by the sectioning processing in space direction.

FIG. 13 is a table showing an example of section information in a caseof sectioning by the sectioning processing in time direction.

FIG. 14 is a table showing an example of section information in a caseof sectioning by the sectioning processing in time/space direction.

FIG. 15 is a table showing an example of divided data in the case of thesectioning processing in space direction as the image sectioningprocessing.

FIG. 16 is a table showing an example of divided data in the case of thesectioning processing in time direction as the image sectioningprocessing.

FIG. 17 is a table showing an example of divided data in the case of thesectioning processing in time/space direction as the image sectioningprocessing.

FIG. 18 is a diagram illustrating an example of an image sectioned bythe sectioning processing in space direction.

FIG. 19 is a flowchart illustrating details of the clarificationprocessing carried out by a clarification processing device.

FIG. 20 is a diagram illustrating an example of a template in a 5×5matrix.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

A description is now given of an embodiment of this invention referringto the drawings.

FIG. 1 is a diagram illustrating a system configuration of an imagegeneration device 101 according to the embodiment of this invention. Theimage generation device 101 includes an image dividing device (alsoreferred to as image processing device) 102, a clarification processingdevice 104, an obfuscation processing device 105, and an imagesuperimposition device (also referred to as latent image embedding unitand a digital watermark embedding unit) 106. The image generation device101 receives, as inputs, input image data 107, information forclarification processing 108, and information for obfuscation processing109, and outputs output image data 112.

The image dividing device 102 includes a sectioning device (alsoreferred to as sectioning unit) 113 and an image partial characteristicanalysis device (also referred to as classification unit) 103. The imagedividing device 102 receives the input image data 107 as an input,generates section information 110, and outputs divided data 111.

The image generation device 101, the image dividing device 102, theimage partial characteristic analysis device 103, the clarificationprocessing device 104, the obfuscation processing device 105, the imagesuperimposition device 106, and the sectioning device 113 may berealized respectively as dedicated devices, or may be realized bygeneral personal computers (PCs) and dedicated software programs.According to this embodiment, an example in which the devices arerealized by general PCs and dedicated software programs is exemplified.

FIG. 2 is a diagram illustrating an example of an internal configurationof the general PC. The PC 201 includes a CPU 202, a memory 203, anetwork interface 204, a keyboard 205, a speaker 206, a display 207, ahard disk 208, and an interface 209.

The CPU 202 is a central processing unit, and executes a program storedin the memory 203 or a program read out from the hard disk 208 into thememory 203.

It should be noted that as necessary, the program may be introduced by astorage medium which is readable by the PC 201, and detachable from thePC 201. In this case, a device for reading the storage medium isconnected to the interface 209, and is then used. As the storage mediumand the device for reading the storage medium, an optical disc (such asCD, DVD, and Blu-ray disc) and a flash memory and a device for using theoptical disc and the flash memory are generally known, and such memoryand device may be used. Moreover, as necessary, the program may beintroduced into the PC via a communication medium (such as acommunication line and a carrier wave on the communication line) by thenetwork interface 204.

The memory 203 is a device for temporarily storing programs executed bythe CPU 202 and data.

The network interface 204 is a device for communicating to/from a deviceoutside the PC 201, such as another PC.

The keyboard 205 is a device operated by an operator of the PC 201 forinputting commands and data to the PC 201.

The speaker 206 is a device for reproducing a signal as sound.

The display 207 is a device for displaying a processing result and thelike.

The hard disk 208 is a device for storing programs and data, and mayinclude a magnetic disk, a non-volatile memory, or the like, forexample. In this case, the programs and data stored in the hard disk 208are normally maintained when the hard disk 208 is turned off, and thenis turned on. It should be noted that an operating system (OS) isintroduced into the hard disk 208 in advance. This configuration enablesspecification of the program through use of a file name, for example. Onthis occasion, the OS is a basic software program for a computer, and awidely known OS may be used.

The interface 209 is used for connection to the devices in the PC 201,and is connected to the respective devices in the PC 201.

As described above, in this specification, the example of realizing theimage generation device 101, the image dividing device 102, the imagepartial characteristic analysis device 103, the clarification processingdevice 104, the obfuscation processing device 105, the imagesuperposition device 106, and the sectioning device 113 (hereinafter,referred to as respective devices) by the PC 201 and the dedicatedsoftware programs is exemplified. Herein, the dedicated software programrefers to a program for controlling the CPU 202 to carry out processing(described later) to be carried out by the respective devices.

Moreover, this specification exemplifies an example of realizing therespective devices by a single PC 201. Data are freely shared among therespective devices through use of a shared area on the memory 203 or thehard disk 208. In a case where the respective devices are realized bythe single PC 201, the dedicated software programs are not necessarilyprovided individually, and the respective devices may be realized bysubroutines (such as functions) in the signal software program asnecessary.

It should be noted that the respective devices may be realized byindependent PCs 201. In this case, communication of data between therespective devices may be carried out via the network interface 204.

A description is now given of processing carried out by the imagedividing device 102 according to this embodiment.

FIG. 3 is a flowchart illustrating a process flow carried out in theimage dividing device 102.

In Step S301, the image dividing device 102 receives the input imagedata 107 as an input. Still image data such as those in JPEG, PNG, PDF,and TIFF formats and the like, and moving image data as those in AVI andMPEG (such as MPEG-2 PS and MPEG-2 TS) formats and the like can be theinput image data 107.

In Step S302, the image dividing device 102 divides the input image data107 into a plurality of sections by using the sectioning device 113. Adescription is given of specific examples of this processing referringto FIGS. 4 to 6 and other figures. As a result of the processing, thesection information 110, which is information relating to the sections,is generated in the memory 203 or the hard disk 208 of the imagedividing device 102. It should be noted that this step is described indetail later.

In Step S303, the image dividing device 102 determines whether or notclassification processing described later has been applied to all thesections divided in Step S302. When the determination is negative inStep S303, the image dividing device 102 takes out a section to whichthe classification processing is not applied from the plurality ofsections divided in Step S302, and proceeds to Step S304.

In Step S304, the image dividing device 102 classifies the sections tobe processed into a clarification-processing-adapted area and anobfuscation-processing-adapted area, and other area as necessary, byusing the image partial characteristic analysis device 103. This step isdescribed in detail later.

On the other hand, when the determination in Step S303 is affirmative,the image dividing device 102 proceeds to Step S305. In Step S305, theimage dividing device 102 unifies the classified results in Step S304,thereby dividing the input image data 107 into theclarification-processing-adapted areas, theobfuscation-processing-adapted areas, and other areas.

In Step S306, the image dividing device 102 outputs the divided data 111which is a result of the division in Step S305. The divided data 111 isdescribed later referring FIG. 15 and the like.

A description is now given of the image sectioning processing carriedout by the sectioning device 113, namely a specific example of theprocessing in Step S302 of FIG. 3. As a first specific example of theimage sectioning processing, a description is given of sectioningprocessing in space direction. The sectioning processing in spacedirection is processing of sectioning a still image into a plurality ofsections without overlaps.

FIG. 18 is a diagram illustrating an example of an image sectionedthrough the sectioning processing in space direction. An image 1801 isdivided into a plurality of sections 1802. In the example illustrated inFIG. 18, the image 1801 is divided into twelve sections arranged in fourcolumns in the horizontal direction, and three rows in the verticaldirection, and all the sections have a common section width and sectionheight.

Arbitrary section width and section height may be specified, but therespective sections are classified into theclarification-processing-adapted area, theobfuscation-processing-adapted-area, and other area, and hence it ispreferred that an area to which the clarification processing is adaptedand an area to which the obfuscation processing is adapted do notcoexist in the same section. Therefore, it is preferred that the sectionwidth and the section height be set to approximately 1 to 10% of theimage width and the image height, respectively, for example.

In the image 1801 illustrated in FIG. 18, there is an area which is notsectioned around the plurality of sections 1802. In other words, a unionof all the sections does not always need to constitute the whole image.

FIG. 4 is a flowchart illustrating an overview of the sectioningprocessing in space direction, which is an example of the imagesectioning processing carried out by the sectioning device 113.

In Step S401, the sectioning device 113 reads the section width and thesection height. The section width and the section height may bespecified in unit of pixel of the image, in unit of an actual size ofthe image, or the like. If the section width and the section height arespecified in unit of the actual size, information on the resolutioncontained in the input image data is used. For example, if the type ofthe image data is TIFF, the image data may have resolution informationas tags, such as ResolutionUnit, XResolution, and YResolution. As theunit, centimeter or inch is usually used, but those units can beproperly converted by the CPU 202 by using the relationship of 1inch=2.54 centimeters. For example, if two centimeters are specified asthe section width, and 300 DPI (300 pixels per inch) are specified asXResolution of the input image, the sectioning device 113 can calculatethe section width in unit of pixel as 300×(2/2.54)=236 pixels (roundedoff to the closest whole number). It should be noted that the sectioningdevice 113 can also calculate the section height in unit of pixel byusing YResolution in a similar manner. The following description isgiven assuming that the section width and the section height arerepresented in unit of pixel.

The section width and the section height may be stored in the hard disk208 within the section device 113 in advance by installation personnelwhen the sectioning device 113 is installed, or when the input imagedata 107 is input, the section width and the section height may also beread together with the input image data 107 by the image generationdevice 101 into the hard disk 208 within the image generation device101, and may be respectively referred to. Moreover, the section widthand the section height may be input from the keyboard 205 of thesectioning device 113 or the like.

In Step S402, the sectioning device 113 acquires int(image width/sectionwidth) and sets int(image width/section width) to the number of sections(horizontal). The number of sections (horizontal) is the number ofsections in the horizontal direction of the image. Moreover, int(x) is afunction for truncating x after the decimal point. This calculation iscarried out per pixel.

In Step S403, the sectioning device 113 acquires int(imageheight/section height) and sets int(image height/section height) to thenumber of sections (vertical). The number of sections (vertical) is thenumber of sections in the vertical direction of the image. Also in thisstep, the calculation is carried out per pixel as in Step S402.

In Step S404, the sectioning device 113 acquires int((image width-numberof sections (horizontal)×(section width)/2) and sets int((imagewidth-number of sections (horizontal)×(section width)/2) to the offset(horizontal). The offset (horizontal) is an offset amount in thehorizontal direction of an image, which is to be used for offsetprocessing carried out in Step S406 described later.

In Step S405, the sectioning device 113 acquires int((image width-numberof sections (vertical)×(section height)/2) and sets int((imagewidth-number of sections (vertical)×(section height)/2) to the offset(vertical). The offset (vertical) is an offset amount in the verticaldirection of the image, which is to be used for the offset processingcarried out in Step S406 described later.

In Step S406, the sectioning device 113 shifts the image by the amountsof the offsets, and sections the image into sections each having thesection width and the section height. The shift of the image by theamounts of offsets means that the sectioning processing starts fromcoordinates (offset (horizontal),offset(vertical)) assuming that theupper left corner of the image data is the origin, the x axis extendsrightward and the y axis extends downward, and the coordinates are setper pixel, as illustrated in FIG. 18.

Moreover, the image is sectioned into the sections through sectioningthe image from the processing start point rightward a number of timescorresponding to the number of sections (horizontal) every section widthand downward a number of times corresponding to the number of sections(vertical) every section height. Each of the sections after thesectioning constitutes a section.

In Step S407, the sectioning device 113 outputs the section information110. The section information 110 includes a section ID 1201, a sectiontype 1202, a start point 1203, an end point 1204, and the like.

In the flowchart illustrated in FIG. 4, it is necessary to carry out theprocessing in Step S402 and the processing in Step 404 in the statedorder. It is also necessary to carry out the processing in Step S403 andthe processing in Step S405 in the stated order. However, the order ofthe processing in Step S402 and the processing in Step S403 may bereversed. In other words, Step S403 may be executed before Step S402,for example.

FIG. 12 is a table showing an example of the section information 110 ina case of sectioning by the sectioning processing in space direction. InFIG. 12, information on one section is described in one row. The sectioninformation 110 is a combination of information on single or a pluralityof sections.

Item names are described in the first row of FIG. 12. The item names areas follows. Specifically, a section ID 1201 is a section ID, a sectiontype 1202 is section type information, and a start point 1203 is startpoint information, and an end point 1204 is end point information.

In FIG. 12, information on respective sections is described in thesecond and the following rows. For example, information on a sectionhaving the section ID 1201 of “1” is described in a row 1205. Thesection type 1202 of the section having the section ID 1201 of “1” isthe space. In other words, the row 1205 represents that the sectionhaving the section ID 1201 of “1” is a section in space direction.

In information on a section having the section type 1202 of “SPACE”, thestart point 1203 is coordinates in an image to constitute a start pointof the section, and the end point 1204 is coordinates obtained by adding+1 both to the x coordinate and the y coordinate of coordinates in theimage to constitute an end point of the section. A rectangular area inwhich the start point and the end point constitute diagonal verticesforms a section. For example, for the section having the section ID 1201of “1” described in the row 1205 of FIG. 12, the start point 1203 is(100, 100), and the end point 1204 is (200, 200). In other words, arectangular area (including borders) having the coordinates (100, 100),coordinates (199, 100), coordinates (199, 199) and coordinates (100,199) as vertices forms a section.

A description is now given of the image sectioning processing carriedout by the sectioning device 113, namely, sectioning processing in timedirection as a second specific example of the processing in Step S302 ofFIG. 3. The sectioning processing in time direction is processing ofsectioning a moving image into frames.

FIG. 5 is a flowchart illustrating an overview of the sectioningprocessing in time direction, which is an example of the imagesectioning processing carried out by the sectioning device 113.

In Step S501, the sectioning device 113 converts the moving image intostill images per frame. The moving image can generally be handled as aset of a plurality of still images (typically 30 or 24 still images persecond). The conversion from a moving image to a set of sill images canbe realized by a generally available function of an OS, a functionprovided by a generally available video extension library, or the like.

In Step S502, the sectioning device 113 adds an I frame out of theplurality of still images acquired by the conversion processing in StepS501 as the section to the section information. Herein, the I framerefers to a frame which can constitute an image without referring toanother frame. In general, the moving image is large in data amount, andhence there is a strong need for the data amount compression.Information on the current image is generally represented by utilizingthe fact that the moving image has a high correlation in the time axisdirection, and thus by utilizing information on images in other frames(namely, other times).

The I frame does not refer to another frame, and it is considered ingeneral that the I frame is high in image quality. Thus, the I frame isto be processed in Step S502.

In Step S503, the sectioning device 113 outputs the section information110.

FIG. 13 is a table showing an example of the section information 110 ina case of sectioning by the sectioning processing in time direction.Item names are described in the first row of FIG. 13. The item names arethe same as those of FIG. 12. For example, information on a sectionhaving the section ID 1201 of “2” is described in a row 1306. On thisrow, there is a description that the section type 1202 is “TIME”. Inother words, the section having the section ID 1201 of “2” is a sectionin time direction.

Moreover, in the information on the section having the section type 1202of “TIME”, the start point 1203 is the frame number to be the startpoint of the section. The information on the end point 1204 is not used,and hence 0 is usually described. It should be noted that the framenumber for the first frame is 0, and the frame number increases one byone subsequently following a frame sequence. For example, for a sectiondescribed on the row 1306, “30” is described as the start point 1203. Inother words, a 30th frame is used as the section.

A description is given above of the sectioning processing in timedirection, but the processing of this embodiment may be carried out pergroup of pictures (GOP). The GOP is a set of still images in a movingimage. Generally, a GOP contains at least one I frame, which constitutesa unit of rewinding or the like. A turnaround time may be reduced bycarrying out the processing from Steps S501 to S503 per GOP.

Moreover, the I frame is to be processed in the above description, butthis is a typical example, and the P frame or the B frame may be to beprocessed. The P frame is a frame referring to the previous frame, andthe B frame is a frame referring to previous and following frames. Itshould be noted that a frame referring to the following frame may be tobe processed. The fact that a frame other than the I frame may be usedalso holds true for the following description.

Moreover, the sectioning is carried out every certain period (everycertain number of frames) in the above-mentioned example, but thesectioning may be carried out at a scene change or a position where animage largely changes. The scene change can be determined by preparing acertain threshold, extracting an I frame, checking whether or not anabsolute value of a difference between this I frame and a previous Iframe (sum of absolute values of differences in each of pixels) islarger than the threshold, and determining a scene change when theabsolute value is larger than the threshold. Moreover, it is possible toemploy another general scene change determination mechanism.

A description is now given of the image sectioning processing carriedout by the sectioning device 113, namely sectioning processing intime/space direction as a third specific example of the processing inStep S302 of FIG. 3. The sectioning processing in time/space directionis processing of sectioning a moving image into frames, extracting an Iframe, and further sectioning the I frame in the space direction.

FIG. 6 is a flowchart illustrating an overview of the sectioningprocessing in time/space direction, which is an example of the imagesectioning processing carried out by the sectioning device 113.

In Step S601, the sectioning device 113 reads the section width and thesection height. This processing is the same as the processing in StepS401 of the flowchart illustrated in FIG. 4, and a detailed descriptionthereof is therefore omitted.

In Step S602, the sectioning device 113 converts the moving image intostill images per frame. This processing is the same as the processing inStep S501 of the flowchart illustrated in FIG. 5, and a detaileddescription thereof is therefore omitted.

In Step S603, the sectioning device 113 extracts an I frame out of theplurality of still images acquired by the conversion processing in StepS602, and shifts the extracted I frame by the amounts of the offsets,and sections the I frame into sections having the section width and thesection height. This processing is realized by applying the processingfrom Step S402 to Step S406 of the flowchart illustrated in FIG. 4 tothe I frame.

In Step S604, the sectioning device 113 outputs the section information110.

FIG. 14 is a table showing an example of the section information 110 ina case of sectioning by the sectioning processing in time/spacedirection. Item names are described in the first row of FIG. 14. Theitem names are the same as those of FIG. 12. For example, information ona section having the section ID 1201 of “3” is described in a row 1407.On this row, there is a description that the section type 1202 is“TIME/SPACE”. In other words, the section having the section ID 1201 of“3” is a section in time/space direction.

In information on a section having the section type 1202 of“SPACE/FRAME”, the start point 1203 describes a frame number n to be astart point of the section, and coordinates p in the frame image in aform of n-p, and the end point 1204 describes coordinates obtained byadding +1 both to the x coordinate and the y coordinate of coordinatesin an n-th frame image to be an end point of the section. A rectangulararea in which the start point and the end point constitute diagonalvertices forms a section. For example, in the row 1407, 30−(100, 100) isdescribed as the start point 1203, and (200, 200) is described as theend point 1204. In other words, a rectangular area (including borders)having the coordinates (100, 100), coordinates (199, 100), coordinates(199, 199), and coordinates (100, 199) as vertices in the 30th frameforms a section.

A description is given of the sectioning processing in time/spacedirection, but the processing of this embodiment may be carried out perGOP, as in the sectioning processing in time direction. A turnaroundtime may be reduced by carrying out the processing from Steps S602 toS604 per GOP.

A description is now given of a specific example of the image partialcharacteristic analysis processing carried out by the image partialcharacteristic analysis device 103, namely a specific example of theprocessing in Step S304 of FIG. 3. First, as an example of the imagepartial characteristic analysis processing, a description is now givenof frequency analysis processing.

FIG. 7 is a flowchart illustrating an overview of the frequency analysisprocessing as the image partial characteristic analysis processingcarried out by the image partial characteristic analysis device 103.

In Step S701, the image partial characteristic analysis device 103 readsan area to be processed of an image to be processed. The image to beprocessed and the area to be processed are specified by the imagedividing device 102. In other words, in Step S303 of FIG. 3, the imagedividing device 102 has extracted a section to be processed, and theextracted section constitutes the area to be processed.

Moreover, the image to be processed is the input image 107 received bythe image dividing device 102 as the input in Step S301 of FIG. 3. Theinput image data 107 is passed and received by using the memory 203 orthe hard disk 208 shared by the image dividing device 102, the imagepartial characteristic analysis device 103, and the sectioning device113.

In Step S702, the image partial characteristic analysis device 103 readsa determination frequency lower limit, a determination frequency upperlimit, and a threshold into the memory 203 of the image partialcharacteristic analysis device 103. It should be noted that those valuesare used for processing in Step S704 described later, and are stored byinstallation personnel or the like in the hard disk 208 of the imagepartial characteristic analysis device 103 as initial setting when theimage partial characteristic analysis device 103 is installed.

In Step S703, the image partial characteristic analysis device 103applies the discrete cosine transform (DCT) to the area to be processedof the image. The DCT is a well-known technology, and is described in K.R. Rao, P. Yip, “Discrete Cosine Transform Algorithms, Advantages,Applications,” Academic Press, Inc.

In Step S704, the image partial characteristic analysis device 103determines whether or not (sum of absolute values of DCT coefficientsfrom determination frequency lower limit to determination frequencyupper limit/(determination frequency upper limit-determination frequencylower limit)) is larger than a threshold. When the determination in StepS704 is affirmative, the image partial characteristic analysis device103 proceeds to Step S705, and when the determination is negative, theimage partial characteristic analysis device 103 proceeds to Step S706.It should be noted that the determination is affirmative in Step S704when there are many low frequency components in the area to be processedof the image, and the determination is negative in Step S704 when thereare many high frequency components in the area to be processed of theimage.

In Step S705, the image partial characteristic analysis device 103 setsa clarification-processing-adapted-area as the determination result.

In Step S706, on the other hand, the image partial characteristicanalysis device 103 sets an obfuscation-processing-adapted area as thedetermination result.

In Step S707, the image partial characteristic analysis device 103outputs the determination result.

It should be noted that, in the above-mentioned processing, two of anupper threshold and a lower threshold as the thresholds may be prepared,and in Step S704, when (sum of absolute values of DCT coefficients fromdetermination frequency lower limit to determination frequency upperlimit/(determination frequency upper limit-determination frequency lowerlimit)) is larger than the upper threshold, the image partialcharacteristic analysis device 103 may proceed to Step S705, when thevalue is smaller than the lower threshold, the image partialcharacteristic analysis device 103 may proceed to Step S706, andotherwise, the image partial characteristic analysis device 103 may set“other area” as the determination result.

A description is now given of template analysis processing as anotherexample of the image partial characteristic analysis processing carriedout by the image partial characteristic analysis device 103. Herein, thetemplate refers to a 3×3 matrix, for example, and is used to extract acharacteristic of an image through convolution with the image. Thetemplate is also called a filter, and typically approximately a 3×3 to7×7 matrix. FIG. 20 is a diagram illustrating an example of the templatein a 5×5 matrix. It should be noted that the template may not be asquare matrix.

FIG. 8 is a flowchart illustrating an overview of the template analysisprocessing as the image partial characteristic analysis processingcarried out by the image partial characteristic analysis device 103.

In Step S801, the image partial characteristic analysis device 103 readsan area to be processed of an image to be processed. This processing isthe same as the processing in Step S701 of the flowchart illustrated inFIG. 7, and a detailed description thereof is therefore omitted.

In Step S802, the image partial characteristic analysis device 103 readsa threshold and a template for partial characteristic determination. Itshould be noted that the threshold and the template for partialcharacteristic determination are used for processing in Step S803described later, and are stored by installation personnel or the like inthe hard disk 208 of the image partial characteristic analysis device103 as initial setting when the image partial characteristic analysisdevice 103 is installed.

In Step S803, the image partial characteristic analysis device 103carries out two-dimensional convolution of the area to be processed ofthe image and the template for partial characteristic determination. Thetwo-dimensional convolution is defined by the following equation (1),where 2w+1 is a width of the template for partial characteristicdetermination, 2h+1 is a height thereof, templ(u,v) is a value of thetemplate for partial characteristic determination at coordinates (u,v),and Img(x,y) is a pixel value at coordinates (x,y) in the area to beprocessed of the image. In the equation (1), c(x,y) represents a resultof the two-dimensional convolution, and has the same size as that of thearea to be processed of the image. In other words, the domain of c(x,y)is also represented as 0≦x<IW and 0≦y<IH, where IW represents the widthof the area to be processed of the image, and IH represents the heightthereof.

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack \mspace{641mu}} & \; \\{{c\left( {x,y} \right)} = {\sum\limits_{u = {- w}}^{w}{\sum\limits_{v = {- h}}^{h}{{{Tmpl}\left( {u,v} \right)}{{Img}\left( {{x - u},{y - v}} \right)}}}}} & (1)\end{matrix}$

It should be noted that coordinates where the upper left corner of thearea to be processed of the image is the origin are used in the equation(1). Moreover, it is assumed that the width and the height of thetemplate are odd numbers in the above description, but the width and theheight may be even numbers. In this case, the width and the height areset to 2w and 2h, respectively. Further, values such as Tmpl(−1,0) andImg(0,−1), which are out of the domain, are set to 0.

In Step S804, the image partial characteristic analysis device 103determines whether or not a sum of the results of the convolutionexceeds a threshold. Herein, the sum of the results of the convolutionrefers to a sum of absolute values of the results of the two-dimensionalconvolution, and is defined by the following equation (2). In theequation (2), |x| represents an absolute value of x, and S representsthe sum of the results of the two dimensional convolution.

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack \mspace{641mu}} & \; \\{S = {\sum\limits_{x = 0}^{{IW} - 2}{\sum\limits_{y = 0}^{{IH} - 1}{{c\left( {x,y} \right)}}}}} & (2)\end{matrix}$

When the determination in Step S804 is affirmative, the image partialcharacteristic analysis device 103 proceeds to Step S805, and when thedetermination is negative, the image partial characteristic analysisdevice 103 proceeds to Step S806. It should be noted that the templateillustrated in FIG. 20 is assumed as the template for the partialcharacteristic determination. Therefore, the determination isaffirmative in Step S804 when there are many low frequency components inthe area to be processed of the image, and the determination is negativewhen there are many high frequency components in the area to beprocessed of the image.

In Step S805, the image partial characteristic analysis device 103 setsan obfuscation-processing-adapted area as the determination result.

In Step S806, on the other hand, the image partial characteristicanalysis device 103 sets a clarification-processing-adapted area as thedetermination result.

In Step S807, the image partial characteristic analysis device 103outputs the determination result.

It should be noted that, in the above-mentioned processing, two of anupper threshold and a lower threshold as the thresholds may be prepared,and in Step S804, when the sum of the results of the convolution islarger than the upper threshold, the image partial characteristicanalysis device 103 may proceed to Step S805, when the sum of theresults of the convolution is smaller than the lower threshold, theimage partial characteristic analysis device 103 may proceed to StepS806, and otherwise, the image partial characteristic analysis device103 may set “other area” as the determination result.

For example, 1 and −1 are arranged in a checkerboard pattern in thetemplate illustrated in FIG. 20. It is considered that a location atwhich the sum of the convolution results with this template is large isa location at which the ups and downs of the image are large, and thislocation is adapted to the obfuscation processing described later.

It should be noted that, in the above-mentioned processing, a templatewhich can be used for searching for a location at which the ups anddowns of an image (such as a 5×5 matrix, all the elements of which is1/25, for example) are small may be prepared as a template, for example,an area in which a sum of convolution results is larger than a thresholdmay be set to a clarification-processing-adapted area, and an area inwhich the sum is not larger than the threshold may be set to anobfuscation-processing-adapted area.

A description is now given of the divided data 111. The divided data 111is data representing a result of dividing the input image 107 into theclarification-processing-adapted area, theobfuscation-processing-adapted area, and the other area based on thesection information 110.

Examples of the divided data 111 are illustrated in FIGS. 15, 16, and17. FIG. 15 is a table showing an example of the divided data in thecase of the sectioning processing in space direction as the imagesectioning processing. FIG. 16 is a table showing an example of thedivided data in the case of the sectioning processing in time directionas the image sectioning processing. FIG. 17 is a table showing anexample of the divided data in the case of the sectioning processing inspace/time direction as the image sectioning processing. In FIGS. 15 to17, divided data in one section is described in one row. The divideddata 111 is a combination of divided data relating to single or aplurality of sections.

In FIGS. 15 to 17, item names are described on the first row as in FIG.12. The item names are as follows. Specifically, a section ID 1201 is asection ID, a section type 1202 is section type information, and a startpoint 1203 is start point information, and an end point 1204 is endpoint information. Those definitions are the same as those in FIGS. 12,13, and 14. The divided data further contains a classification 1501representing whether this section is the clarification-adapted area orthe obfuscation-adapted area. The item of the classification 1501 mayfurther contain a content “OTHER AREA.”

In FIGS. 15 to 17, divided data for each section is described startingfrom the second row. For example, information on a section having thesection ID 1201 of “1” is described in FIG. 15. The section type 1202 ofthe section having the section ID 1201 of “1” is space, the start point1203 thereof is (100, 100), the end point 1204 thereof is (200, 200),and the classification 1501 thereof is the clarification-adapted area.In other words, it is represented that the section in a rectangular area(including borders) having the coordinates (100, 100), coordinates (199,100), coordinates (199, 199) and coordinates (100, 199) as vertices is aclarification-processing-adapted area. This holds true for FIGS. 16 and17, and a detailed description thereof is thus omitted.

A description is now given of the processing carried out by the imagegeneration device 101.

FIG. 9 is a flowchart illustrating a process flow carried out in theimage generation device 101.

In Step S901, the image generation device 101 receives input image data107 as an input. As described above, still image data such as those inJPEG, PNG, PDF, and TIFF formats and the like, and moving image datasuch as those in AVI format and the like can be the input image data107.

In Step S902, the image generation device 101 divides the input imagedata 107 into the clarification-processing-adapted areas, theobfuscation-processing-adapted areas, and other areas by using the imagedividing device 102. The details of this processing are the same asthose described above in the description of the image dividing device102.

In Step S903, the image generation device 101 applies the clarificationprocessing to the information for clarification processing 108 by usingthe clarification processing device 104, thereby generating an imageafter clarification processing. This processing is described in detaillater.

In Step S904, the image generation device 101 applies the obfuscationprocessing to the information for obfuscation processing 109 by usingthe obfuscation processing device 105, thereby generating an image afterobfuscation processing. This processing is described in detail later.

In Step S905, the image generation device 101 superimposes the imageafter clarification processing created in Step S903 on theclarification-processing-adapted area of the input image data 107 byusing the image superimposition device 106. This processing is describedin detail later.

In Step S906, the image generation device 101 superimposes the imageafter obfuscation processing created in Step S904 on theobfuscation-processing-adapted area of the input image data 107 by usingthe image superimposition device 106. This processing is described indetail later.

In Step S907, the image generation device 101 outputs the image dataacquired by the processing in Step S906 as the output image data 112.The format of the output image data 112 is the same as the format of theinput image data 107.

It should be noted that the processing of Steps S901, S902, S903, S905,and S907 in the process flow should be carried out in the stated order.Similarly, the processing of Steps S901, S902, S904, S906, and S907 inthe process flow should be carried out in the stated order.

When the input image data 107 is a moving image, and the processing iscarried out per GOP, in the above-mentioned two orders, Step S907, forexample, for a previous GOP may be carried out before Step S901, forexample, for a next GOP.

On the other hand, the other orders are arbitrary in the above-mentionedprocess flow. For example, the order of the processing in Step S903 andthe processing in Step S904 may be reversed.

Moreover, in a case where the image is a color image in theabove-mentioned process flow, the image may be separated into abrightness component and other components in Step S901, for example, andthe processing in Steps S902 to S906 may be applied to the brightnesscomponent. In this case, a color image is reconstructed by thebrightness component to which the processing up to Step S906 has beenapplied, and the other components separated in Step S901, and is outputin Step S907.

For example, in a case where the input image data is an RGB image, it ispossible to extract the brightness component Y from the components ofthe RGB image in accordance with the following equation (3). On thisoccasion, C_(b) and C_(r), which are the color-difference signals can beacquired as the other components by calculation. Moreover, therespective components of the RGB image can be calculated from thebrightness component Y and the color-difference signals C_(b) and C_(r)in accordance with the following equation (4). Each of thosecalculations is carried out for each pixel.

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack \mspace{641mu}} & \; \\{\begin{bmatrix}Y \\C_{b} \\C_{r}\end{bmatrix} = {\begin{bmatrix}0.299 & 0.587 & 0.114 \\{- 0.168736} & {- 0.331264} & 0.5 \\0.5 & {- 0.418688} & {- 0.081312}\end{bmatrix}\begin{bmatrix}R \\G \\B\end{bmatrix}}} & (3) \\{\left\lbrack {{Math}.\mspace{14mu} 4} \right\rbrack \mspace{641mu}} & \; \\{\begin{bmatrix}R \\G \\B\end{bmatrix} = {\begin{bmatrix}1 & 0 & 1.402 \\1 & {- 0.344136} & {- 0.714136} \\1 & 1.772 & 0\end{bmatrix}\begin{bmatrix}Y \\C_{b} \\C_{r}\end{bmatrix}}} & (4)\end{matrix}$

It should be noted that the color-difference signals C_(b) and C_(r) maytake negative values in the equations (3) and (4). Moreover, the valueafter calculation in accordance with the equation (4) is preferablyproperly adjusted so as to be an integer from 0 to 255 in a typicalexample. For example, the value after the calculation in accordance withthe equation (4) is rounded off, truncated, or rounded up to an integer.Moreover, for example, adjustment of setting the value to 0 when thevalue after the calculation in accordance with the equation (4) is anegative value, and setting the value to 255 when the value is equal toor more than 256 is carried out.

As described above, the image generation device 101 creates the outputimage data 112 enhanced in an effect of preventing unauthorized leak.

A description is now given of the clarification processing by theclarification processing device 104.

A copy preventing pattern causing letters “DO NOT COPY” to appear for aprinted matter such as a certificate is prevailing. The clarificationprocessing is processing of causing an image created from theinformation for clarification processing 108, which is described later,to appear on a copied image as the copy preventing pattern when theimage is copied.

It should be noted that the image does not appear when the image issimply copied because electronic data thereof are the same. However,scale-down processing is often applied to an image. For example, for astill image, a scaled-down image for a list display called thumbnail maybe created. Moreover, for a moving image, the image width and the imageheight may be reduced for a reduction in data amount. If this processingis applied, the electronic data is different between the image beforethe scale-down and the image after the scale-down. The clarificationprocessing is carried out by using this difference. In other words, theclarification processing is processing of embedding an image forclarification processing representing information such as “DO NOT COPY”into the image as a latent image, and the embedded image forclarification processing appears when the scale-down processing isapplied to the image.

First, a description is now given of the information for clarificationprocessing 108 referring to FIG. 10. FIG. 10 illustrates variousexamples of the information for clarification processing 108 accordingto the embodiment of this invention. Item names are described in thefirst row in FIG. 10. An information type for clarification processingis described in an information type 1001, and an example of theinformation for clarification processing is described in an example ofinformation for clarification processing 1002.

In FIG. 10, the information for clarification processing 108 isexemplified in the second and subsequent rows. One piece of informationfor clarification processing 108 is exemplified in one row. For example,information for clarification processing having the information type1001 of “IMAGE” is described in a row 1003. In this case, a file name ofdata representing image data such as “/image/hideImage.jpg” is describedin the information for clarification processing 108. Based on this filename, the CPU 202 in the clarification processing device 104 transmits arequest to an OS, thereby acquiring the image data from the hard disk208 in the clarification processing device 104.

Moreover, the information having the information type 1001 of “SENTENCE”is described in a row 1004 as an example of another piece of theinformation for clarification processing 108. In this case, a sentence,a used font name, a size of the used font, and display coordinates, suchas “DO NOT COPY”, “GOTHIC”, 36, (100, 100)”, are specified to theinformation for clarification processing 108. A fact that a sentence “DONOT COPY” in 36-point gothic font is drawn starting from the coordinates(100, 100) is specified in the example in the row 1004.

Moreover, the information having the information type 1005 of “SCRIPT”is described in a row 1005 as an example of another piece of theinformation for clarification processing 108. In this case, a scriptsuch as “PutString(100, 100), GetDate( )+“for”+GetUserName( ) “MINGTYPE”, 48” is specified to the information for clarification processing108. In the example in the row 1005, a result of drawing a date and auser name in 48-point Ming-type font at coordinates (100, 100)constitutes the information for clarification processing. It should benoted that the script is a simple program, and many types of softwareprograms as an execution engine (software program for executing ascript) for a script language are generally prevailing. A scriptlanguage becomes available by storing an arbitrary script executionengine in the hard disk 208 of the clarification processing device 104.

FIG. 19 is a flowchart illustrating the clarification processing carriedout by the clarification processing device 104, namely detailedprocessing in Step S903 illustrated in FIG. 9.

In Step S1901, the clarification processing device 104 carries outinitial setting. In the initial setting, the clarification processingdevice 104 reserves an area for the image after clarification processingto be output on the memory 203, and reads the information forclarification processing 108, thereby creating the image forclarification processing.

For example, when the information type 1001 of the information forclarification processing 108 is image, the clarification processingdevice 104 reads this image data as the image for clarificationprocessing. Moreover, for example, when the information type 1001 of theinformation for clarification processing 108 is sentence, theclarification processing device 104 creates an image for clarificationprocessing obtained by drawing a sentence in accordance with theinformation for clarification processing 108. This processing can becarried out through use of functions of the OS. Moreover, for example,when the information type 1001 of the information for clarificationprocessing 108 is script, an image for clarification processing iscreated by executing a script for the information for clarificationprocessing 108 through use of the script execution engine.

It should be noted that the image for clarification processing isadjusted so as to have the same image width and the image size as thoseof the input image data 107. The adjustment can be carried out by amethod of simply cutting off an oversized portion, or adding white datain a lacking portion, for example. Moreover, an image may be copied sothat the same information for clarification processing 108 periodicallyrepeats in the lacking portion.

In Step S1902, the clarification processing device 104 determineswhether or not the processing in Step S1904 or Step S1905 is applied toall the pixels of the image for clarification processing created in StepS1901. When the clarification processing device 104 determines that theprocessing has been applied to all the pixels of the image for theclarification processing, the clarification processing device 104outputs data generated in the area reserved on the memory 203 for theimage after clarification processing as the image after clarificationprocessing, and finishes the clarification processing. On the otherhand, when the clarification processing device 104 determines that theprocessing has not been applied to all the pixels of the image for theclarification processing, the clarification processing device 104extracts an unprocessed pixel from the image for clarificationprocessing, and proceeds to Step S1903.

In Step S1903, the clarification processing device 104 determineswhether or not the pixel extracted in Step S1902 is within a remainingarea. Herein, the remaining area refers to an area constituted by, whenan image is scaled down at a given scale-down rate, pixels influencingpixel values of an image after the scale-down.

In a case where the nearest-neighbor interpolation method is used as thescale-down processing, there are pixels in the image before thescale-down which do not affect the image after the scale-down. This isbecause, in the nearest-neighbor interpolation method, an image afterthe scale-down is constituted by values of a certain number of pixelsout of the image before the scale-down, and the number of pixels in theimage after the scale-down decreases, compared with the image before thescale-down. Thus, an operation which is intuitively understood asthinning out pixels is carried out.

It is possible to determine whether or not a pixel to be processed(before scale-down) is within a remaining area according to thefollowing equation, provided that (x,y) represents a position of thepixel, a (<1.0) represents a scale-down ratio, c1 and c2 representinteger constants, and int(x) represents a function giving the maximuminteger equal to or less than x.

When integers (u,v) which satisfy both int((2u+1)/2a)+c/=x andint((2u+1)/2a)+c2=y exist, and (u,v) is at a pixel position to becontained in the image after the scale-down, the pixel (x,y) is within aremaining area.

It should be noted that the integer constants c1 and c2 take values from0 to approximately 15, for example, and may be the same value or may bedifferent values. The integer constants c1 and c2 are parametersproviding for a case in which a remaining area is translated inparallel, and, as an example, the integer constants c1 and c2 may bechanged from 0 to 15 with an increment of 1 in the time direction or thespace direction of a moving image, or may be changed by randomlyselecting a value therebetween. In this way, even if a remaining areamoves in parallel, a portion where a latent image appears can becreated. It should be noted that c1 or c2 may take a larger value or asmaller value, for example, 100, −15, or −100. The integer constants c1and c2 may be stored in advance in the hard disk 208 of theclarification processing device 104 by installation personnel or thelike when the clarification processing device 104 is installed.

In Step S1903, when it is determined that the position of pixel to beprocessed is in a remaining area, the clarification processing device104 proceeds to Step S1904, and when it is determined that the positionof pixel to be processed is not in a remaining area, the clarificationprocessing device 104 proceeds to Step S1905.

In Step S1904, the clarification processing device 104 assigns(corresponding pixel value of image for clarificationprocessing+corresponding pixel value of input image data 107−constant)to a value of the pixel to be processed. The constant is, for example,an average of pixel values of all the pixels of the image forclarification processing. When the calculation result is negative, thevalue of pixel to be processed is set to 0, and when the calculationresult exceeds a permissible maximum value (typically 255) for a pixelvalue, it is preferred that the pixel value to be processed be set tothe maximum value (typically 255).

Moreover, before the assignment to the pixel value of the pixel to beprocessed, the corresponding pixel value of the image for clarificationprocessing may be changed. For example, an intensity image which has thesame size as that of the input image data 107, and in which each pixeltakes a value in a range from 0 to 200, for example, is further input tothe clarification processing device 104, and a pixel value of a pixel ofthe intensity image corresponding to a subject pixel of the input imagedata 107 (a pixel value of a pixel of the intensity image correspondingto a pixel position before the scale-down) is multiplied by acorresponding pixel value of the image for clarification processing, andthen divided by 100. The obtained value may serve as a correspondingpixel value of the image for clarification processing when theassignment to a pixel value of a pixel to be processed is carried out.With this configuration, the intensity of a latent image can be setbased on the position.

On the other hand, in Step S1905, the clarification processing device104 assigns the corresponding pixel value of the input image data 107 toa value of the pixel to be processed.

In the above-mentioned processing, it is possible to generate an imageafter clarification processing which concentrates information on theimage for clarification processing on remaining areas, and which doesnot reflect the information on the image for clarification processing toareas other than the remaining areas.

A description is now given of the clarification processing by theobfuscation processing device 105.

A description is now given of information for obfuscation processing 109used for the obfuscation processing, referring to FIG. 11. FIG. 11illustrates various examples of the information for obfuscationprocessing 109 according to the embodiment. Item names are described inthe first row in FIG. 11. An information type for obfuscation processingis described in an information type 1101, and an example of theinformation for obfuscation processing is described in an example ofinformation for obfuscation processing 1102.

In FIG. 11, the information for obfuscation processing 109 isexemplified starting from the second rows. One piece of information forobfuscation processing 109 is exemplified in one row. For example,information for obfuscation processing having the information type 1101of “SENTENCE” is described in a row 1103. In this case, very short(typically four characters in Japanese, and eight letters in alphabet)sentence such as “Rent0001” is described in the information forobfuscation processing 109. “Rent0001” means to be lent to a person ofthe ID number “0001”.

Moreover, information for obfuscation processing having the informationtype 1101 of “INTEGER” is described in a row 1104. In this case,typically, a 64-bit integer such as “0x123456789ABCDEF0” is described inthe information for obfuscation processing 109. It should be noted that“0x123456789ABCDEF0” represents a 64-bit integer in the notation of theC language.

Moreover, for example, information for obfuscation processing having theinformation type 1101 of “SCRIPT” is described in a row 1105. In thiscase, a script such as “GetDate( )<<32+GetUserID( )” is described in theinformation for obfuscation processing 109. It should be noted that thescript is described above. A 64-bit integer obtained as a result ofexecuting this script constitutes the information for obfuscationprocessing 109.

A description is now given of the obfuscation processing carried out bythe obfuscation processing device 105. This obfuscation processing istypically performed by embedding a digital watermark, which is generallyknown. The information for obfuscation processing 109 is embedded in theinput image data 107 by a method of embedding a digital watermark asdescribed in US 2002/0007403, for example, the content of which ishereby incorporated by reference into this application.

On this occasion, when the input image data 107 is a still image, adigital watermark is directly embedded, thereby generating an imageafter obfuscation processing. Moreover, when the input image data 107 isa moving image, a digital watermark is embedded into only a framedetermined as an obfuscation-processing-adapted area in the divided data111, thereby generating an image after obfuscation processing. Then, 0may be assigned to portions which are not determined as theobfuscation-processing-adapted area in the divided data 111.

A description is now given of the processing of Step S905 of FIG. 9carried out by the image superimposition device 106 out of thesuperimposition processing carried out by the image superimpositiondevice 106. First, the image superimposition device 106 refers to thedivided data 111, and extracts one or a plurality of sections having theclassification 1501 of the clarification-processing-adapted area. Then,the image superimposition device 106 extracts area information on thesection from the start point/end point information of this section, andreplaces input image data 107 in this area by an image of acorresponding area of the image after clarification processing. On thisoccasion, replacement means an assignment of a pixel value.

It should be noted that when the processing of Step S906 is carried outbefore the processing of Step S905, “the input image data 107 in thisarea” to be replaced is the image data after the replacement processingin Step S906.

A description is now given of the processing of Step S906 of FIG. 9 outof the superimposition processing carried out by the imagesuperimposition device 106. First, the image superimposition device 106refers to the divided data 111, and extracts one or a plurality ofsections having the classification 1501 of theobfuscation-processing-adapted area. Then, the image superimpositiondevice 106 extracts area information on the section from the startpoint/end point information of this section, and replaces image dataafter the replacement processing in Step S905 in this area by an imageof a corresponding area of the image after obfuscation processing. Onthis occasion, replacement means an assignment of a pixel value.

It should be noted that when the processing of Step S906 is carried outbefore the processing of Step S905, the input image data 107 in theobfuscation-processing-adapted area is replaced by an image of acorresponding area of the image after obfuscation processing.

The image sectioning processing by the image dividing device 102provides an effect that it is difficult to remove a result of theobfuscation processing such as the digital watermark, compared with thesimple area sectioning (such as sectioning through the use of bitplanes), and a description is now given of other effects.

There is also provided an effect that the image quality of the outputimage data 112 improves, compared with a case where the image sectioningprocessing is not carried out by the image dividing device 102 and onlythe clarification processing is carried out by the image generationdevice 101. The clarification processing has more adverse effects interms of image quality on the output image data 112, compared with theobfuscation processing. However, it is possible to reduce the adverseeffects in terms of the image quality by controlling the image dividingdevice 102 to carry out the image sectioning processing, and by limitingsubjects to which the clarification processing is applied toclarification-processing-adapted area.

For the purpose of using only the above-mentioned effects, theobfuscation-processing-adapted area may be treated in the same way asthe other areas, thereby omitting the processing. Moreover, theobfuscation processing device 105 may not be included in the imagegeneration device 101 in the first place, and Step S305 of FIG. 3 may beconfigured to divide the input image data intoclarification-processing-adapted areas and other areas. In this case,the determination result is set to the other area in Step S706 of FIG. 7and Step S805 of FIG. 8.

According to the above described image generating device of theembodiment, based on the characteristic of the image, the plurality ofsections to be processed are classified into at least theclarification-processing-adapted areas for embedding a latent image intothe image, and the obfuscation-processing-adapted areas for embedding adigital watermark into the image. As a result, the image can beclassified into the area suitable for embedding a latent image, and anarea suitable for embedding a digital watermark based on thecharacteristic of the image.

In particular, appropriate classification processing can be carried outby classifying a plurality of sections to be processed intoclarification-processing-adapted areas andobfuscation-processing-adapted areas based on the frequencycharacteristic of an image. Specifically, an area having a large numberof high frequency components is classified into anobfuscation-processing-adapted area, and it is thus possible to preventvisibility of an image from degrading by applying the obfuscationprocessing to the obfuscation-processing-adapted area. Moreover, an areahaving a large number of low frequency components is classified into aclarification-processing-adapted area, and it is thus possible to causea latent image to effectively appear when the scale-down processing isapplied to an image in which the clarification processing is applied tothe clarification-processing-adapted area.

Appropriate classification processing can be carried out by carrying outconvolution of image data and template data for partial characteristicdetermination for each of a plurality of sections to be processed, andclassifying the plurality of sections to be processed into theclarification-processing-adapted area and theobfuscation-processing-adapted area based on a result of theconvolution.

If the image is a still image, the still image is sectioned into theplurality of sections to be processed. If the image is a moving imageconstituted by a plurality of frames, a part of the plurality of framescontained in the moving image is extracted as the plurality of sectionsto be processed. Moreover, in a case of a moving image constituted by aplurality of frames, some of the plurality of frames constituting themoving image are extracted, and each of the extracted frames may besectioned into a plurality of sections to be processed. As a result ofthe processing, irrespective of whether an image is a still image or amoving image, the image can be classified into aclarification-processing-adapted area and anobfuscation-processing-adapted area.

An image generation device of an embodiment embeds the latent image intothe clarification-processing-adapted area, and embeds the digitalwatermark into the obfuscation-processing-adapted area. As a result,when the scale-down processing is applied to an image, it is possible tocause a latent image to effectively appear. Moreover, it is possible tomake removal of a latent image and a digital watermark by a third persondifficult by embedding the latent image and the digital watermark in theareas based on the characteristic of the image.

The embodiments of this invention have been described in detailreferring to drawings, but the specific configuration is not limited tothese embodiments and may include other designs without departing fromthe gist of this invention.

1. An image processing device, comprising: a sectioning unit forsectioning an image into a plurality of sections to be processed; and aclassification unit for classifying the plurality of sections to beprocessed into at least a clarification-processing-adapted area forembedding a latent image into the image, and anobfuscation-processing-adapted area for embedding a digital watermarkinto the image based on a characteristic of the image.
 2. The imageprocessing device according to claim 1, wherein the classification unitclassifies the plurality of sections to be processed into theclarification-processing-adapted area and theobfuscation-processing-adapted area based on a frequency characteristicof the image.
 3. The image processing device according to claim 1,wherein the classification unit carries out convolution of image dataand template data for partial characteristic determination for each ofthe plurality of sections to be processed, and classifies the pluralityof sections to be processed into the clarification-processing-adaptedarea and the obfuscation-processing-adapted area based on a result ofthe convolution.
 4. The image processing device according to claim 1,wherein: the image is a still image; and the sectioning unit sectionsthe still image into the plurality of sections to be processed.
 5. Theimage processing device according to claim 1, wherein: the image is amoving image containing a plurality of frames; and the sectioning unitextracts a part of the plurality of frames contained in the moving imageas the plurality of sections to be processed.
 6. The image processingdevice according to claim 1, wherein: the image is a moving imagecontaining a plurality of frames; and the sectioning unit extracts apart of the plurality of frames contained in the moving images, andsections each of the extracted frames into a plurality of sections to beprocessed.
 7. An image generation device, comprising: the imageprocessing device according to claim 1; a latent image embedding unitfor embedding the latent image into the clarification-processing-adaptedarea; and a digital watermark embedding unit for embedding the digitalwatermark into the obfuscation-processing-adapted area.
 8. An imageprocessing method, comprising: sectioning an image into a plurality ofsections to be processed; and classifying the plurality of sections tobe processed into at least a clarification-processing-adapted area forembedding a latent image into the image, and anobfuscation-processing-adapted area for embedding a digital watermarkinto the image based on a characteristic of the image.
 9. The imageprocessing method according to claim 8, wherein the classifyingcomprises classifying the plurality of sections to be processed into theclarification-processing-adapted area and theobfuscation-processing-adapted area based on a frequency characteristicof the image.
 10. The image processing method according to claim 8,wherein the classifying comprises carrying out convolution of image dataand template data for partial characteristic determination for each ofthe plurality of sections to be processed, and classifying the pluralityof sections to be processed into the clarification-processing-adaptedarea and the obfuscation-processing-adapted area based on a result ofthe convolution.
 11. The image processing method according to claim 8,wherein: the image is a still image; and the sectioning comprisessectioning the still image into the plurality of sections to beprocessed.
 12. The image processing method according to claim 8,wherein: the image is a moving image containing a plurality of frames;and the sectioning comprises extracting a part of the plurality offrames contained in the moving image as the plurality of sections to beprocessed.
 13. The image processing method according to claim 8,wherein: the image is a moving image containing a plurality of frames;and the sectioning comprises extracting a part of the plurality offrames contained in the moving images, and sectioning each of theextracted frames into the plurality of sections to be processed.
 14. Animage generation method, comprising: sectioning an image into aplurality of sections to be processed; classifying the plurality ofsections to be processed into at least aclarification-processing-adapted area for embedding a latent image intothe image, and an obfuscation-processing-adapted area for embedding adigital watermark into the image based on a characteristic of the image;embedding the latent image into the clarification-processing-adaptedarea; and embedding the digital watermark into theobfuscation-processing-adapted area.